Retrieve information sooner with DataNOW!
Server-based report writer speeds reporting for HP 3000 applications, databases
by
Shawn M. Gordon
President
S.M.Gordon & Associates

I remember about 10 years ago there were a dozen or more report writers out there for the HP 3000. Everything from end-user reporting to ways to make programmers more productive. Some of these were fourth-generation languages such as PowerHouse and SpeedWare, some were code generators like Protos and Gus.

Today with ODBC and client-server, the idea of server-based report writers has fallen out of favor. Many of those products have disappeared now, but DataNOW! managed to survive that whole time for a couple of very important reasons. It is very fast to develop reports in, and the reports execute very quickly.

DataNOW! is menu driven and has extensive online help, so the learning curve for basic reporting is light.

How does it work?

DataNOW! started back on the Classic MPE computers when everyone was looking for ways to improve performance. A standard technique used was MR NOBUF reads, which stands for Multi-Record Non-Buffered. This allowed you to read more than one record at a time with each FREAD and then deblock the record in memory. The performance gains from this technique were amazing, and formed the basis of other products such as QueryCalc, SuperTool and SortMagic.

What is interesting is that MR NOBUF still delivers performance gains on the PA-RISC 3000s, even though everyone said that wouldn’t be true because of pre-fetching.

From what I understand this has to do with the fact that you are making less calls to the file system, and so the overhead is lower. If you have a million records to read, you can do it with 100,000 reads instead of a million.

DataNOW! uses this MR NOBUF technique, as well as some sophisticated sorting algorithms, to achieve the greatest performance possible.

I ran comparative tests against some Quiz reports to get an idea of the performance gain you see from DataNOW! In one instance a Quiz report ran in 105 minutes, and the DataNOW! report ran in 35 minutes. In another test the Quiz report wasn’t finished after 9 hours and the DataNOW! report ran in 75 minutes.

Features

As I mentioned earlier, DataNOW! is menu driven, so you are prompted for everything as you go. If you don’t know how to answer a question, just hit the function key to pop up the help on the topic. You can easily do a basic query in DataNOW! without looking at the manual at all, other than to find out what program to run.

Take a look at the examples below to see a sort of standard dialog to enter a database name, select a dataset, and select the data items. You can stack up to six databases delimited by commas, and then choose as many datasets and data items as you want from a menu or by entering them yourself.

Examples

DataNOW!---- 6:43 AM--------------------------------------Demo for Shawn Gordon
(C) Copyright 1989-1995 by CosmoSoft Corporation. All rights reserved. 

------------------------------ SELECT DATA BASES ---------------------Screen 10
DATA BASES: TRACE.PUB.SMGA
PASSWORD? (TRACE.PUB.SMGA):

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +++ ++

DataNOW!---- 6:44 AM--------------------------------------Demo for Shawn Gordon
BASE: TRACE.PUB.SMGA

------------------------ SELECT SETS BY NAME OR NUMBER ---------------Screen 20
SETS? (TRACE.PUB.SMGA):

---------------------------------- SET MENU -----------------------------------
1 A:INNER-JOB-A 4 M:OUTER-JOB-M 7 D:RUN-STAT-D 
2 A:RUN-STAT-A 5 M:ALIAS-M 8 D:ALIAS-D
3 A:STREAMER-A 6 D:INNER-JOB-D

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +++ ++++++++++++

DataNOW!---- 6:45 AM--------------------------------------Demo for Shawn Gordon
BASE: TRACE.PUB.SMGA SET: M:OUTER-JOB-M ENT: 68 

----------------------- SELECT ITEMS BY NAME OR NUMBER ---------------Screen 30
ITEMS? (RETURN=ALL):

---------------------------------- ITEM MENU ----------------------------------
PREFIX A. SET 4 M:OUTER-JOB-M (TRACE.PUB.SMGA) 68 ENT. 

1 U26 OUTER-JOB 3 X60 JOB-DESC(1) 5 X60 ..(3) 2 X6 FLAGS 4 X60 ..(2)

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +++ ++

DataNOW!---- 6:45 AM--------------------------------------Demo for Shawn Gordon
BASE: TRACE.PUB.SMGA SET: M:OUTER-JOB-M ENT: 68 

--------------------- SPECIFY FIND RULES FOR FIND PASS 1 -------------Screen 50
THESE ITEMS ARE NOW IN THE DISPLAY:
A.1 U26 IOUTER-JOB A.4 X60 JOB-DESC(2)
A.2 X6 FLAGS A.5 X60 JOB-DESC(3)
A.3 X60 JOB-DESC(1)

FIND RULE? (RETURN=DONE): OUTER-JOB <> " " 


DataNOW!---- 6:46 AM--------------------------------------Demo for Shawn Gordon
BASE: TRACE.PUB.SMGA SET: M:OUTER-JOB-M ENT: 68 FOUND: 68 RECORD 4 OF 68 IN FIND LIST
---------------------------- FIND 1 KEY SERIAL -------------------Screen 40
OUTER-JOB 
FLAGS < >
JOB-DESC(1) <

JOB-DESC(2) <

JOB-DESC(3) <

There are numerous options in DataNOW! for creating temporary fields, calculated fields, converting data between character and integer types, manipulating strings, and applying boolean logic. The logic on variables looks a lot like what is used in Quiz.

You can prompt for input at execution time, save your current sessions as an XEQ file and do date handling — and DataNOW! even knows about MANMAN and Chronos dates. You can create basic join rules between data structures, but if it’s not a key value, then you are going to have to make a pass to create a KSAM file first.

The output formatting in DataNOW! can be anything from very basic extraction of items to complex customer invoice-style reports. The layout is kind of interesting, because you can visually lay out the page, putting keywords into it to dictate where and what to break, sort, and total on. I was really pleasantly surprised on the reporting options. I don’t know why, but I had always assumed that DataNOW! didn’t have very good reporting ability.

Installation and Documentation

Installation is easy and proceeded without incident. The manual is about 160 pages and includes a table of contents and an index. The beginning of the manual has a quick tutorial to get you started, and the rest is clearly laid out to correspond with the product. The rest of the manual has chapters dedicated to explaining how to do extracts, report formatting, configuring, etc. The manual is easy to read, and complete. But you should probably scan through it once before you try to get too deep into DataNOW! so you don’t waste time trying to do things the wrong way.

Conclusions

DataNOW! isn't the most glamorous looking HP 3000 application, but it is a very functional one. When using tools such as DataNOW! and comparing them to using a language like COBOL, you have to decide where you pay. With COBOL you pay up front in time to develop, but typically you have an efficient report. With PowerHouse it’s cheap up front, but you typically pay every time you run the program. With a product like DataNow! you only pay while learning to use it. Once you know how to use it, it’s fast to develop and very fast to execute — much faster than anything you are going to write in a 3GL, let alone a 4GL. I think you will be astounded by the performance of DataNOW! and its robust report generation capability.

There is really only one thing that I would like to see improved. Currently DataNOW! can’t link non-key items between datasets. It has to make a pass at the data and generate a KSAM file. I talked to the developer, and he said he wants to change this soon.

If you are sick of paying upgrade fees and high support prices for products, and really want to speed up your production, you owe it to yourself to check out DataNOW! It’s the most bang for your buck I have seen in a long time.